Skip to content

[WIP] feat(falco): experimental multi-thread support#3833

Draft
irozzo-1A wants to merge 10 commits intofalcosecurity:masterfrom
irozzo-1A:experimental/multi-thread
Draft

[WIP] feat(falco): experimental multi-thread support#3833
irozzo-1A wants to merge 10 commits intofalcosecurity:masterfrom
irozzo-1A:experimental/multi-thread

Conversation

@irozzo-1A
Copy link
Copy Markdown
Contributor

@irozzo-1A irozzo-1A commented Mar 22, 2026

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

/kind release

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area engine

/area tests

/area proposals

/area CI

What this PR does / why we need it:

The libs branch with multi-thread support can be found here.

To build assuming the two repositories are in the same directory, use the following:

cd falco
cmake -B build \
  -DUSE_JEMALLOC=ON \
  -DUSE_GPERFTOOLS=ON \
  -DBUILD_BPF=OFF \
  -DBUILD_DRIVER=OFF \
  -DBUILD_LIBSCAP_MODERN_BPF=ON \
  -DENABLE_E2E_TESTS=OFF \
  -DUSE_BUNDLED_DEPS=ON \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DFALCOSECURITY_LIBS_SOURCE_DIR=$(pwd)/../libs \
  . && cmake --build build --target falco -j$(nproc)

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:


@poiana
Copy link
Copy Markdown
Contributor

poiana commented Mar 22, 2026

Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@poiana
Copy link
Copy Markdown
Contributor

poiana commented Mar 22, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: irozzo-1A
Once this PR has been reviewed and has the lgtm label, please assign fededp for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Iacopo Rozzo and others added 9 commits April 10, 2026 07:21
…ed buffers

- Add num_worker_threads to modern_ebpf config (falco.yaml, configuration, schema)
- Open live inspector with buffers_num = num_worker_threads when > 1
- Validate syscall buffer count for multi-thread in configure_syscall_buffer_num
- Add push_ctrl_msg() to falco_outputs for workers to push pre-formatted messages
- Implement do_inspect_multi_thread() with per-worker engines, formats, buffer handles
- Add design proposal 20251205-multi-thread-falco-design-review.md
- Engine: filter_details_resolver and filter_macro_resolver AST visitor updates

Made-with: Cursor
Signed-off-by: Iacopo Rozzo <iacopo.rozzo@localhost>
Align libsinsp open_modern_bpf() with the new iters_num parameter. In
multi-thread mode set iters_num to num_workers so each worker can use BPF
iterator paths concurrently; single-buffer mode uses DEFAULT_ITERS_NUM.

Signed-off-by: irozzo-1A <[email protected]>
Replace gmtime() with gmtime_r() when formatting event times for JSON output.
gmtime is not thread-safe; parallel worker threads calling format/output paths
triggered TSAN data races on the static tm buffer.

Signed-off-by: irozzo-1A <[email protected]>
Calling httplib Server::Get() after listen() races with worker threads
handling other routes (TSAN). Register /metrics during start() with the
other routes, and gate the handler with an atomic until inspectors are open
(503 until ready). enable_prometheus_metrics() only flips the ready flag.

Signed-off-by: irozzo-1A <[email protected]>
Runs Falco TSAN build with 8 workers under stress-ng and event-generator.
Enables metrics and webserver Prometheus endpoint, scrapes /metrics in the
background, and uses configurable RUN_DURATION_SEC (default 600s).

Signed-off-by: irozzo-1A <[email protected]>
Gate the multi-threaded worker code path behind the FALCO_MULTI_THREAD
CMake option (OFF by default). When disabled, the worker_context and
do_inspect_multi_thread code is excluded at compile time and sinsp is
built with single-threaded sync policy (zero-cost no-op mutexes).

When enabled (-DFALCO_MULTI_THREAD=ON), the option forwards
ENABLE_MULTI_THREAD to libs, activating real mutexes and Folly
ConcurrentHashMap for thread-safe sinsp state.

Signed-off-by: irozzo-1A <[email protected]>
…ic race reports

Assemble a combined suppressions file from both Falco-specific and libs
suppressions at runtime. Add report_atomic_races=0 to TSAN_OPTIONS to
suppress Folly hazptr atomic-vs-mutex-init false positives that cannot
be matched by suppressions on both stacks.

Signed-off-by: irozzo-1A <[email protected]>
@irozzo-1A irozzo-1A force-pushed the experimental/multi-thread branch from 0bc2209 to 91db364 Compare April 13, 2026 12:53
@github-actions
Copy link
Copy Markdown

This PR may bring feature or behavior changes in the Falco engine and may require the engine version to be bumped.

Please double check userspace/engine/falco_engine_version.h file. See versioning for FALCO_ENGINE_VERSION.

/hold

Omit engine.modern_ebpf.num_worker_threads from the embedded JSON schema and
reject non-zero values at load time when libsinsp is built without
ENABLE_MULTI_THREAD. Use a two-step stringify so the schema macro expands
before stringification.

Sync CMake cache ENABLE_MULTI_THREAD with FALCO_MULTI_THREAD using FORCE so
toggling the Falco option reliably rebuilds libs.

Remove num_worker_threads from the default falco.yaml and document the optional
key in comments to avoid schema warnings on non-multi-thread builds.

Signed-off-by: irozzo-1A <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

2 participants